The Annals of Probability 

2004, Vol. 32, No. 3A, 2029-2052 

DOI: 10.1214/009117904000000676 

© Institute of Mathematical Statistics, 2004 

LIMIT THEOREMS FOR A CLASS OF IDENTICALLY 
DISTRIBUTED RANDOM VARIABLES 

By Patrizia Berti, Luc a Pratelli and Pietro Rigo 

Universita' di Modena e Reggio-Emilia, Accademia Navale di Livorno 
and Universita' di Pavia 

A new type of stochastic dependence for a sequence of random 
variables is introduced and studied. Precisely, (X n )n>i is said to be 
conditionally identically distributed (c.i.d.), with respect to a filtra- 
tion (Gn) n >Q, if it is adapted to {Q n ) n >o and, for each n > 0, (X k ) k> „ 
is identically distributed given the past Q n . In case Go = {0,Q} and 
Gn = a(Xi, . . . ,X n ), a result of Kallenberg implies that (X n ) n >i is 
exchangeable if and only if it is stationary and c.i.d. After giving some 
natural examples of nonexchangeable c.i.d. sequences, it is shown that 
(X„) n >i is exchangeable if and only if (X T ( n )) n >i is c.i.d. for any fi- 
nite permutation r of {1, 2, . . .}, and that the distribution of a c.i.d. 
sequence agrees with an exchangeable law on a certain sub-cr-field. 
Moreover, (1/n) Xws=i ^ k conver g es a - s - an d in L 1 whenever (X n ) n >i 
is (real-valued) c.i.d. and _E[|A"i|] < oo. As to the CLT, three types 
of random centering are considered. One such centering, significant 
in Bayesian prediction and discrete time filtering, is E[X n +i \Q n ]- For 
each centering, convergence in distribution of the corresponding em- 
pirical process is analyzed under uniform distance. 

1. Introduction and motivations. In this paper a new type of stochas- 
tic dependence for a sequence (X n ) n >i of random variables is introduced 
and studied. Precisely, suppose the X n are defined on the probability space 
(Q,A, P), take values in the measurable space (E,£), and are adapted to 
a filtration Q = (£? n ,) n >o- Then, {X n ) n is said to be conditionally identically 
distributed with respect to Q, abbreviated as Q-c.iA., whenever 

E[f(X k )\g n ] = E[f(X n+1 )\g n ] a.s. 

for all k > n > and all bounded measurable / : E — > M. 
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Roughly speaking, (1) means that, at each time n > 0, the future observa- 
tions (Xfc)fc>n are identically distributed given the past Q n . In case Q = Q x , 
where Q x = {0, and Q x = (j{X\ , X n ), the filtration is not mentioned 
at all and (X n ) n is just called c.i.d. Clearly, if (X n ) n is (/-c.i.d., then it is 
c.i.d. and identically distributed. 

Two obvious equivalent formulations of (1) are 

X k ~X n+1 under P(-\H) 

for all k> n > and all events H e G n with P{H) > 0, 

where "~" means "distributed as," and 

(E[/(X n+ i)|g n ]) n > is a (/-martingale 

for every bounded measurable /:£"—> M. 

By general results on martingales, condition (3) can be written as E[f(XT+i)] = 
E[f(Xi)] for all bounded measurable / and all finite (/-stopping times T 
(where (/-stopping times take values in {0, 1, ... , oo}). Say that a (/-stopping 
time S is predictable in case S = T + 1 for some (/-stopping time T. Then, 
one more equivalent formulation of (1) is 

(4) Xs ~ X\ for each finite predictable (/-stopping time S. 
Note also that, when Q = Q x , conditions (l)-(4) all reduce to 

(5) [X 1 ,...,X n , X n+2 ] ~[X u ...,X n , X n+1 ] for all n > 0. 

Exchangeable sequences meet (5) and, thus, are c.i.d. Indeed, exchange- 
ability is the most significant case of conditional identity in distribution. 
C.i.d. sequences, however, need not be exchangeable. In fact, by a remark- 
able result of Kallenberg [(1988), Proposition 2.1], exchangeability amounts 
to stationarity and condition (5). In Kallenberg's paper (cf. Proposition 
2.2), it is also shown that conditions (3)-(5) are equivalent in case Q = Q x . 
However, apart from these results, condition (5) is not systematically inves- 
tigated. 

In the present paper, instead, we focus on (/-c.i.d. sequences. As a first 
motivation, we give some examples where conditional identity in distribution 
naturally arises while exchangeability may fail. 

Example 1.1 (Stopping and sampling). Let X n = Ztau, where (Z n ) n is 
exchangeable and T is a random variable with values in{l,2,...,oo}. Then, 
(X n ) n is not exchangeable apart from trivial cases, but it is c.i.d. under nat- 
ural conditions on T. In fact, if (Z n ) n is c.i.d. (and not necessarily exchange- 
able), then {X n ) n is c.i.d. whenever T is independent of (Z n ) n , or whenever 
T is a predictable stopping time for Q z . Thus, typically, conditional iden- 
tity in distribution is preserved under stopping while exchangeability is not. 
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We now prove that (X n ) n is c.i.d. if (Z n ) n is c.i.d. and T is a predictable 
stopping time for Q z . If S is a finite predictable stopping time for Q z , then 
T A S is a finite predictable stopping time for Q z , and since (Z n ) n is c.i.d., 
one obtains 

X s = Ztas ~ Zi = X\. 

Since (X n ) n is adapted to Q z , condition (4) implies that (X n ) n is G z -c.Ld. 
and, in particular, it is c.i.d. Next, conditional identity in distribution is 
also preserved under (strictly increasing) sampling. That is, if ( y Z n ^ n is c.i.d. 
and Ti < T2 < ■ ■ • are finite predictable stopping times for Q z , then (X n ) n = 
(ZT„) n is c.i.d. To prove the latter fact, fix a finite predictable stopping time 
S for Q x . Since T\ < T2 < ■ ■ ■, one has {Tj =n,S = j} G Gn-i f° r an h n > 1j 
and this implies {T5 = n} = IJ^j^jTj = n, S = j} G It follows that T5 

is a finite predictable stopping time for Q z , and since (Z n ) n is c.i.d., one 
obtains 

Xg = Zt s ~ ^1 ~ ^Ti = ^1 • 

If (X n ) n is stationary and converges in probability, then X n = X\ a.s. 
for all n. In Example 1.1, if T is a.s. finite, then (X n ) n = (2rAn)n is defini- 
tively constant with probability 1 and, thus, it converges a.s. but in a trivial 
way. The next example exhibits a c.i.d. (nonexchangeable) sequence which 
converges a.s. in a nontrivial way. 

Example 1.2 (Compensated sum of independent random variables). Given 
the real numbers < b\ < 62 < ^3 < • ■ • < c, let us define 7^ = c and 7^ = 
hi A 6j for i ^ j. On noting that T n = {"fij)i<ij< n is a symmetric positive 
definite matrix, {X n ) n can be taken such that [X\, . . . ,X n ] ~AA(0,T n ) for 
each n > 1. Then, 

[Xi, . . . , X n , X n+2 ] ~ Af(0, T n+1 ) for all n > 0, 

that is, (X n ) n is c.i.d. [by condition (5)]. However, 

E[(X n -X m ) 2 ]=2(c-b n Ab m ). 

Thus, (X n ) n is not stationary unless b n = b\ for all n, and E[(X n — X) 2 ] — > 
if c = lim n b n , for some random variable X . Further, X n —> X a.s. whenever 
J2n( c ~ bn) r < 00 for some r > (since — X\ 2r ] = 7 r (c — b n ) r for some 

constant j r ). To explain the title of the example, we note that it is a partic- 
ular case of the following general scheme. Let (Z n ) n , (U n ) n be independent 
sequences of independent real random variables and let 
n 

X n = ^2Zi + U n , G n = a(Z 1 ,Ui,...,Z n ,U n ), G O = {0,£1}. 

i=l 
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Suppose also that U n compensates Ya=i m the sense that X n ~ X\ for 
all n, and that the characteristic function <pxi of X\ is null on a set with 
void interior. Fix k > n > and a bounded Borel function / : R — > R. Then 



where is the distribution of — X)?=i ^i- But ^ = fj, n +i, due to <£xi is 
null on a set with void interior and, thus, (X n ) n is Q-c.i.d. For instance, given 
any nondegenerate and infinitely divisible law /i, the sequences (Z n ) n and 
(U n ) n can be taken such that the resulting (X n ) n is c.i.d., nonexchangeable 
with X\ ~ [i. Finally, to recover the first part of the example, just take 
Z n ~ AA(0, b n — b n -i) and U n ~ AA(0, c — 6 n ), where 6o = 0. 

Example 1.3 (Modified Polya urns). An urn contains w > white and 
r > red balls. At each time n > 1, a ball is drawn and then replaced together 
with d n more balls of the same color. Let X n be the indicator of the event 
{white ball at time n}. Then E(X\) = w/(w + r) and 



In the usual Polya scheme, d n = d\ for all n, where d\ > 1 is a fixed integer, 
and (X n ) n turns out to be exchangeable. Here, instead, we let (d n ) n be 
any sequence of random variables, with values in {1,2,...}, satisfying the 
following: 

(i) d n is independent of a(Xi,dj :i<n,j<n) for all n > 1, or 

(ii) d\ is degenerate and a(d n ) C cr(Xi, . . . ,X n -i) for all n > 2. 

Then (X n ) n is c.i.d. but (apart from particular cases) nonexchangeable. 
For instance, if all the d n are degenerate, {X n ) n is not exchangeable unless 
d n = d\ for all n. To prove that (X n ) n is c.i.d., it is enough to check that 
(E[X n+ i\G n ]) n >o is a (/-martingale for some filtration Q D Q x . Suppose (i) 
holds and let 

G o = {0,tt}, Q n = a(Xi,d 1 ,...,X n ,d n ,d n+1 ) for n > 1. 

For n = 0, (i) implies S[£7[X 2 |^i]|^ ] = E[X 2 ] = E[Xx] = E[Xi\G ] a.s. For 
n > 1, (i) gives £?[X n+ i|£ n ] = E[X n+1 \Xi, di, . . . ,X n , d n ] a.s. Since d n+ i is 
(/^-measurable, it follows that 




i=l 



a.s. 



E[X n+ i\Xi, di, . . . ,X n ,d n ] 



w + E?=id i X l 
w + r + Y%=i di 



a.s. for all n > 1. 



£[£[X n+2 |0„+i]|0„] 



^ + S"=l rfi^i + ^n+l^[^n+l|gn] 

w + r + Y,Uld l 
E[X n+1 \g n ] a.s. 



A similar argument works under (ii) , after setting Q = Q 
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There is a second reason for studying Q-c.i.d. sequences, in addition to 
their possible utility in modelling real phenomena. Indeed, conditional iden- 
tity in distribution is a basic assumption in uniform limit theorems for pre- 
dictive inference and empirical processes from dependent data. 

Precisely, suppose E is a Polish space, £ = B(E) and (X n ) n is any se- 
quence of random variables. Given a class T> of bounded measurable func- 
tions on E, let 

a n (f) = E[f(X n+1 )\g n ] for all / € V, 

be the so-called predictive measure. In various problems, mainly in Bayesian 
predictive inference, discrete time filtering and sequential procedures, the 
main goal is just evaluating a n , and good approximations a n for a n are 
needed. See, for instance, Algoet (1992, 1995), Ould-Said (1997), Modha and Masry 
(1998), Berti and Rigo (2002) and Berti, Mattei and Rigo (2002). Usually, 
a n is asked to meet a consistency condition of the type supj g:D \a n (f) — 
a n(f)\ a.s. A further request is that, for suitable normalizing con- 
stants c n , the limiting distribution of c n (a n — a n ) can be evaluated. Here, 
c n (a n — a n ) is viewed as a process (indexed by V) with paths in l°°(V), 
the space of bounded functions on T> equipped with uniform distance; see 
van der Vaart and Wellner (1996). In this framework, possible choices for 
a n and c n are the empirical measure \i n = - Ya=i and c n = \fn. So, it is 
of some interest to give conditions for 

(6) sup|/i n (/) -a n (f)\ — >-0 a.s. 
fev 

(7) \fn{[i n — cin) converges in distribution to some known limit. 

Now, assuming that (X n ) n is Q-c.i.d. is fundamental for both (6) and (7). 
As to (6), we refer to Berti, Mattei and Rigo (2002). As to (7), one of the 
concerns of this paper is proving it for Q-c.i.d. sequences; see Section 4. Note 
also that (7) implies (6) if a.s. convergence is weakened to convergence in 
probability. 

To sum up, conditional identity in distribution seems interesting enough 
to deserve a systematic study, both from the theoretical and the applied 
points of view. This task is accomplished here from the first point of view, 
with special attention to limit theorems. 

The paper is organized in three sections. In Section 2, a few basic facts are 
listed. Among other things, a c.i.d. sequence meets a SLLN, is asymptotically 
exchangeable, and its probability distribution agrees with an exchangeable 
law on a certain sub-a-field of £°°. Moreover, (X n ) n is exchangeable if and 
only if (X T ( n )) n is c.i.d. for any (finite) permutation r of {1,2,...}. Sec- 
tion 3 includes versions of the CLT for Q-c.i.d. sequences. Let / :E — > R 
be a measurable function. Stable convergence (in particular, convergence in 
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distribution) of v^tnEti/W — L n ) is investigated for three different 
choices of the random centering L n . In particular, conditions are given for 
convergence in distribution of 



,(/)) = E / W - E[f(x n+1 )\g n yj . 



Such conditions, incidentally, work in Examples 1.2 and 1.3. Section 4 is de- 
voted to uniform limit theorems. For each centering considered in Section 3, 
convergence in distribution of the corresponding empirical process is investi- 
gated under uniform distance. General statements for Q-c.i.d. sequences are 
obtained which, among other things, yield interesting (and possibly new) 
results in the particular case of exchangeable sequences. 

2. Preliminary results and the SLLN. Let 7i be the class of measurable 
functions f:E^>M. such that _E[|/(Xi)|] < oo. Our starting point is the 
following simple lemma. 

Lemma 2.1. Let {X n ) n be Q-c.i.d. Then, for each f £ TC, there is an 
integrable random variable Vf such that E[f(X n+ i)\Q n ] — > Vf , a.s. and in 
L 1 , and 

(8) E[V f \g n ] = E[f(X n+1 )\G n ] a.s. for every n > 0. 

Moreover, if fi, ■ ■ ■ , fk ore bounded elements ofTC, k > 1, then 



(9) 



E 



n fj(x n +j, 
■j=i 



Qn 



k 



a.s. and in L 1 . 



Proof. By (3), {E[f (X n+ i)\Q n ]) n >o is a (/-martingale, and it is uni- 
formly integrable since the X n are identically distributed. Hence, E\f{X n+ i)\Q r , 
Vf, a.s. and in L , for some random variable Vf. In particular, Vf closes the 
martingale (E[f ' {X n+ i)\Q n ]) n >Q and, thus, condition (8) holds. As to (9), 
since fi, ■ ■ ■ , fk are bounded, it is enough to show a.s. convergence. Arguing 
by induction, suppose that 



E 



k-l 



■3=1 



Qn 



k-l 
3=1 



a.s. 



Let D n = E[(V fk -E[V fk \Q n }) U^Zl fj(X n+j )\G n ]. Since f\,...J k are bounded 
and E[Vf k \G n ] — > Vf k a.s., it follows that D n — > a.s. Hence, (8) and the in- 
ductive assumption imply 



E 



■3=1 
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fc-1 

I fj{X n +j)E[fk(X n+k )\G n+k _i] 
Lj=l 

7c— 1 

U fj(X n+j )V f) ft, 
J'=l 



'fc-1 



II fji X n+j, 

■i=i 



£77 



a.s. 



□ 



Among other things, Lemma 2.1 has implications as regards convergence 
in o~(L l ,L°°) of c.i.d. sequences. Recall that, for real integrable random vari- 
ables Y n and Y on the same probability space, Y n — > y in a{L l ,L°°) means 
E\Y n Z] — ► E[YZ] for each bounded random variable Z. Then, f(X n ) —>Vf in 
o^L-^L 00 ) whenever (X n ) n isQ-c.i.d. and f &H. Fix, in fact, a bounded ran- 
dom variable Z. By standard arguments, for proving E[Z f {X n )\ — > E[ZVf] 
it can be assumed that Z is <5 m -measurable for some m, and in this case 
Lemma 2.1 yields 

£7[Zy^] =lim£;[Z£;[/(X n )|a n _ 1 ]] = 15m£[Z/(X n )]. 
Moreover, in exactly the same way, Lemma 2.1 also implies that 



J'=l 



fc 



in^L 1 ,!, 00 ) 



whenever (X n ) n is £/-c.i.d. and /i, • • • , /fc are bounded elements of From 
now on, when (X n ) n is c.i.d. and / G"H, Vf always denotes a version of the 
limit in <t(L 1 ,L°°) of{f{X n )) n . 

If (X n ) n is C?-c.i.d., then (/(^T n )) n is still £/-c.i.d. for each measurable 
/ on E 1 , while (g(X n , X n+ i, . . .)) n can fail to be c.i.d. if g is measurable 
on E°°; see, for instance, Example 1.2. Nevertheless, (g(X n ,X n+ i, . . . )) n 
obeys a SLLN for various choices of g, for instance, for g of the form g(x) = 

nti/jfo), x = ( Xl ,x 2) ...)eE°°. " 



Theorem 2.2 (SLLN). Let (X n ) n be c.i.d. Iffi,...,fk £^ and /i, . . . , / fe _i 
are bounded, k>l, then 

-i n— 1 fc fc 

( io ) -En u (Xi+j ) - n y h a - s - md in l1 ■ 

n i=0j=l j=l 

In particular, ^ Ya=i f{X%) — ► V/, a.s. and in L 1 , whenever f G7i. 
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PROOF. Let Ui = Y[j =1 fj(X i+ j), i>0. Since (Ui)i is uniformly inte- 
grable, it is enough to prove a.s. convergence, and, to this end, it can be 
assumed fj > for all j. To begin with, suppose also that fk is bounded, 
and let 



Ui- Epilog] 



i + 1 



Then, (Z n ) n is a martingale with respect to (^ +fe _ 1 ) n , and since fi,- ■ ■ , fk 
are all bounded, one has sup n £'[Z^] < oo. Hence, (Z n ) n converges a.s., and 
an application of the Kronecker lemma gives 



i n— 1 



n . 

i=0 



For k = 1, one has ± E[Ui\g^ h _ x ] = \ E^ 1 E[h(X i+1 )\gf] - V fl a.s. 
by Lemma 2.1 and, thus, (11) implies (10). Arguing by induction, suppose 
that 



^ n— 1 fc— 1 fc— 1 



( i2 ) -En»-n^ 



a.s. 

n i=0 j=l j=\ 



Recall that, if (a n ) n and (b n ) n are any real sequences, then ^ ^27=0 a «^« — * a ^ 
whenever ^ X)£o a i^ a i b n ^-b and cjj > for all i. Hence, 



n— 1 



5>wS fc _i] 



n . 

i=0 



-, n— 1 fc— 1 fc 

-En /,(^ + ,)^[/fc(^ + fc)i^fc-i] - n v h 

i=0 j=l i=l 



a.s. 



Once again, (10) follows from (11), and this concludes the proof in the 
particular case where fk is bounded. If fk is not bounded, define fk, m = 
fkl{f k <m} - Then, Vf k m | Vf k a.s. as m — ► oo. Further, for each fixed mo, one 
obtains 



Vfr > limSUpS[/fc,m(^m+fc)|^+fc-l] 

m 

> liminf E[f Km {X m +k)\Gm+k-i\ 

> liminf £;[/ fc|mo (X m+fe )|a^ +fe _ 1 ] = F/ femo a.s. 

Thus, y /fe = lim m ^[/fc, m (X m+fc )|^ +fc _ 1 ] a.s. Let Y { = f k ,ii X i+k) x IlJ=i /iPQ+y)- 
Since /i, . . . , /fc-i are bounded, condition (12) holds by the first part of this 
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proof. Therefore, 

-, n— 1 n— lk— 1 



i=0 «=o j=l 

Us 



a.s. 



Further, E^o^i / *i) < E,~o > < 1 + £[/fc(*i)] whi ch im- 

plies P(C/i / Y { i.o.) = 0. Hence, it suffices showing that £Ef=c l( Y i ~ -^1 
- ► a.s. In its turn, this follows from the Kronecker lemma, after 

noting that H n = Er=o "~ i+i +k ~ 1 * s a martingale such that sup n E[H 2 ] < 

oo. In fact, letting a = YljZi SU P l/j|> one obtains 

OC 



s a pE[H%< 1 £ i Tr 2 EiYl_ 1 ] 

n=l 

oo , n _i 

< 2a 2 V n~ 2 / xP(f k (Xi) > x) dx 



71=2 

oo n—1 



< 2o 2 £ 71" 2 £ iP(/ fc (Xi) > i - 1) 

n=2 i=l 

oo oo 

= 2a 2 ]Ti J P(/ fc (X 1 )>i-l) £ 
i=l n=i+l 



n- 2 



OO 



<2a 2 ^P(/ fc (X 1 )>f-l) 



i=i 



<2a 2 (l + E[f k (X 1 )]). □ 

Remark 2.3. Suppose E is a Polish space, £ = 13(E) and (X n ) n is 
c.i.d. By using Theorem 2.2 and Lemma 2.4, it is not hard to see that 
^ Ya=i g(Xi,Xi+i, . . . ) converges a.s. for each bounded continuous function 
g on E°°. This result generally fails if g is a bounded, Borel but not contin- 
uous function on E°° . 



The remaining part of this section investigates to what extent conditional 
identity in distribution is connected with exchangeability. To this end, we 
collect here some notation and terminology from Aldous (1985). Given a 
Polish space S, let Cb(S) denote the space of bounded continuous functions 
on S, P the set of probability measures on B(S), and S the a-field on P gen- 
erated by the evaluation maps p^>p(B), for B varying in B(S). A random 
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measure on S is a measurable function 7 : (£7, ^4) — > (IP, S). Let (Z n ) n be a se- 
quence of S- valued random variables on (Q,A, P). Say that (Z n ) n converges 
stably if, for every H £ A with P(H) > 0, (Z n ) n converges in distribution 
under P(-\H) to some law [in. In this case, there is a random measure 
7 on 5 which represents each limit law fin as //#(•) = / "f(u;)(-)P(du\H), 
and (Z n ) n is said to converge stably with representing measure 7. See also 
Letta and Pratelli (1996). We recall that, if {Z n ) n is exchangeable, there is 
a random measure 7 on S such that the product random measure 



is a version of the conditional distribution of (Z n ) n , given £7(7). Such 7 is 
called the directing measure of (Z n ) n . For our purposes, a last simple fact 
should be stressed. 

Lemma 2.4. Let S be a Polish space and (Z n ) n any sequence of S -valued 
random variables on (£l,A, P). If a(Z\, Z2, ■ ■ ■ ) C cr(\J n Q n ) and (E[f (Z n+ \)\ 
Gn])n converges a.s. for each f £ Cb(S), there is a random measure 7 on S 
such that (Z n ) n converges stably with representing measure 7 and 



PROOF. Fix H G Q m with P(H) > 0, m > 1. Then, E P{ .\ H) [f(Z n )] con- 
verges to a finite limit for all / G Cb(S), so that (Z n ) n converges in distribu- 
tion under P(-\H). Since cr(Zi, Z2, . . . ) C o"(lJ n <5 n ), it follows that (Z n ) n con- 
verges in distribution under P(-\H) for each H € A with P(H) > 0. Hence, 
there is a random measure 7 on 5 such that (Z n ) n converges stably with rep- 
resenting measure 7. Such 7 can be taken o~(Zi, Z2, ■ ■ ■ )-measurable. Let / € 



C b (S) a ndL f = lim n E[f(Z n+1 )\g n ] a.s. Then, E[I H L f ] = E[I H f /(x) 7 (-)(^)] 



for all H E \J n Gn- Since and / f{x)^(-){dx) are measurable with respect 
to a(U n £/ n ), one obtains Lj — J /(x)7(-)(<ix) a.s. D 

Suppose E is a Polish space, 8 = 13(E) and (X n ) n is £?-c.i.d. In view 
of Lemmas 2.1 and 2.4, there is a random measure a on such that Vf = 
f f(t)a(-)(dt) a.s. for all / G Cb{E). By a monotone class argument, it follows 
that 



whenever B £ £. In the sequel, a is called the directing measure of (X n ) n . 
Such terminology, which is typical of exchangeable sequences, is motivated 
by at least two facts. First, by Theorem 2.2, a is the a.s. weak limit of 
empirical measures, 



7 = 7 x 7 x 




a.s. for each f G Cb{S). 



V lB {u) = a{u){B) 



for almost all lv 



1 



n 



2_^^Xi(w) * ol{u) weakly, for almost all ui. 

i=l 
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Second, (X n ) n is asymptotically exchangeable and the exchangeable limit 
law is, in a sense, directed by a. In fact, something more is true in the 
following: 

Theorem 2.5. Suppose E is a Polish space, £ = B(E) and (X n ) n is Q- 
c.i.d. Then, [X n , X n+ i, . . .] converges stably with representing measure a 00 , 
where a is the directing measure of {X n ) n , and 

E[g(X n+1 ,X n+2 ,...)\g n ] 

(13) 



g(x)a°° (-){dx) a.s. for each g G C b {E°°). 

Proof. For each n > 0, fix a regular version v n of the conditional dis- 
tribution of [X n+ i, X n+ 2, ■ ■ ■] given Q n , and define 

v n ^(oj){B) = v n (uj){x G E°° G B} for all k > n, to G SI and B G £■■ 

Let /i n = f n ,n+i- Since (X n ) n is Q-c.i.d. and is a version of the condi- 
tional distribution of Xk given Q n , one obtains v n ^ = fj, n a.s. for all k > n > 0. 
By Lemma 2.1, / f(t)fi n (-)(dt) -> / f(t)a(-)(dt) a.s. for each / G C b (E), and 
this implies fJ> n (^) —> ct{u)) weakly for almost all u. Let H\ G A be such that 
P{H\) = 1 and, for all u> G ifi, (// n (w)) n is tight and ^^(w) = /x n (u;) for all 
> n > 0. Then, (f n (u;)) n is tight, too, for all cj G Hi. Let Do C Cb(E) be 
a countable convergence determining class for £\ and let D be the class of 
those functions g on E°° of the form g(x) = Yij=i fj( x j)> x = ( x i,%2, ■ ■ ■ ) £ 
B 00 , where fc > 1 and A, . . . , / fc G D . For 5 G £>, say g(x) = Uj=i fj(xj), 
Lemma 2.1 gives 

k n k n 

g(x)a 00 {uj){dx) = J] / fj(t)a(u)(dt) = J[ V fj (u) = lim / g(x)u n (u)(dx) 

j=l J 3=1 

for almost all w. Since -D is countable, there is H 2 G A with P(H2) = 1 
and / g(x)o! 00 (a;)(d3;) = lim n J g(x)v n (uj){dx) for all g € D and G i?2- It 
follows that u n {hj) — > a°°{tju) weakly for all u> G Hi n i^2- This proves (13). 
To conclude the proof, it suffices to apply Lemma 2.4 with S = E°° and Z n = 
\X n , X n+ i ,...]. 

□ 



By Theorem 2.5, [X n ,X n +i, . . .] converges in distribution under P(-\H) to 
the exchangeable law //#(•) = / a°°(u})(-)P(du>\H) whenever (X n ) n is c.i.d., 
H G A and P{H) > 0. An alternative proof of this fact could be given by 
results of Aldous (1985). Incidentally, we also note that Theorem 2.5 directly 
implies Kallenberg's result that (X n ) n is exchangeable if and only if it is 
stationary and c.i.d.; see Section 1. In fact, if {X n ) n is stationary and c.i.d., 
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the distribution of [X n ,X n+ i, . . .] does not depend on n (by stationarity) 
and converges weakly to the exchangeable law fin [by Theorem 2.5, since 
(X n ) n is c.i.d.]. 

One more consequence of Theorems 2.2 and 2.5 is that any c.i.d. law on 
£°° is exchangeable on a suitable sub-cr-field. Let 7r n be the nth coordinate 
projection on E°° and V the cr-field on E°° generated by 

limsup — [/(vri) + • • • + /(vr n )] for all bounded /6H. 

n n 

Theorem 2.6. Suppose E is a Polish space, £ = B{E) and (X n ) n is 
c.i.d. Then, the probability distribution A of {X n ) n coincides on V with the 
exchangeable law «(•) = / a°°(uj)(-)P{duj), where a is the directing measure 
of (X n ) n . 



Proof. By Theorem 2.5, [X n ,X n+ i, . . .] converges in distribution to u. 
Given /i, ...,/& £ Cb(E), this fact and Lemma 2.1 imply 



3=1 



II fj( X n+j) 
■3=1 



E 



■ k 
■3=1 



Let Vo be the cr-field on E°° generated by limsup n ^[/(7Ti) H h /(vr n )] for 

all / £ Cb(E), and let h be any product of generators of Vo, that is, 

k 1 n 

h = TT limsup - V fj o TTj where /i, . . . , f k £ C 6 (£). 

By Theorem 2.2, limsup„ ± Y%=i fj(Xi) = V f . a.s. and, thus, jhd\ = E\R% =1 V fj ]. 
On the other hand, exchangeability of fi implies 

k -I n 
3=1 *=1 



fc n 



Jhdfi = J limf[^jTf j on i dv = lim^ J {[J^fj o^dfi 



k V k 1 

J ]lf j oK j dn = E \\\ f = J 
3=1 y=i -I 



i=i i=i 
hdX. 



Hence, A = (i on Vo- To conclude the proof, it is sufficient showing that 
VCcr(VoUAA), where = {A e £°° : A(yl) = = 0}. Let i/ = (A + u)/2. 
Given a bounded measurable <f> on E and e > 0, there is / E Cb{E) such that 
/ |</>(7Ti) — /(vti)| dv < e. Since (vr n ) n is c.i.d. under u, Theorem 2.2 implies 



~ 1 n 1 n 

/ limsup - ^(7Tj) - limsup -^/(7Tj 

J n n . -. n n . 

1=1 1=1 



dv 



LIMIT THEOREMS 



13 



lim 

n 



1 



$>fa)-/fa)) 



n t , 

i=i 



du 



< y |^(7ri)-/(7ri)|di/<e. 
This concludes the proof. □ 



Remark 2.7. In Theorem 2.6, V cannot be replaced by the shift-invariant 
(T-field of (n n ) n . In fact, if the distribution of {X n ) n agrees with an exchange- 
able law (i on the shift-invariant <r-field of (7r n ) n , then 

P(3\\mX n ) = /i(31im-7r n ) 

= li(3 m with 7r n = ir m for all n > m) 
= P(3 m with X n = X m for all n > m), 

where the second equality is due to exchangeability of fx. But, there are c.i.d. 
sequences for which P(3limX n ) = 1 > = P(3m with X n = X m for all n > 
m), for instance, the one exhibited in Example 1.2. It follows that, unlike 
in the exchangeable case, the shift-invariant cr-field of (vr n ) n and V have not 
the same completion under an arbitrary c.i.d. law on £°°. 



We close this section with a characterization of exchangeability in terms 
of conditional identity in distribution. 

Theorem 2.8. Let X be the shift-invariant a -field of (X n ) n . The fol- 
lowing statements are equivalent: 

(i) (X n ) n is exchangeable; 

(ii) For any He! with P(H) > 0, (X n ) n is c.i.d. under P{-\H); 

(iii) For any (finite) permutation r o/{l,2, ...}, {X T / n \) n is c.i.d. 

PROOF, (i) => (ii). Obvious, (ii) => (iii). Fix f & H and note that, by The- 
orem 2.2, Vf can be taken X-measurable. Hence, by (ii), E[f(Xi)\X\ = Vf a.s. 
Further, let n > 1, HE Q% and K £ X. For all k> n, condition (ii) implies 

E[f(X k )I H I K ] = E[f(X n+l )I H I K ). 

Since f(Xf.) — > Vf in a{L l ,L°°) as k — ► oo, one also obtains EIVjIhIr] = 
Since Vf is T-measurable, this implies 

E[f(X n+ i)\I] = V f = E[f(X n+1 )\IvgX] a.s. 

Thus, (X„) n is exchangeable and, clearly, this implies condition (iii). 
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(iii)=^(i). We prove that, for any m,...,n p distinct integers, any r > 1 
and any m > max(ni , . . . , n p ) , 



(14) 



E 



n/,-(x m+j )n^(^n ( ) 

Lj=l 



1=1 



E 



n/jpwn^) 



Z=l 



where /i, . . . ,f r ,gi, ■ ■ ■ ,g P are bounded elements of TL. We argue by induction 
onp> 1. When p = 1, condition (14) follows from applying (5) to the c.i.d. 
sequence (X r ( n )) n , where r is a finite permutation such that r(j) =m + j 
for j = 1, . . . , r, r(r + 1) = m and r(r + 2) = 1. Suppose now that (14) holds 
for some p. We have to prove 



(15) 



E 



k p+1 
Lj=l 1=1 



E 



' k p+1 
UfiiXrr+AUgiiXi) 
3=1 1=1 



for any k > 1 and any m > max(ni, . . . ,n p , n p ±i). Let t\ denote a finite 
permutation such that t\ (j) = m + j for j = 1, . . . , fc, t\ (k + /) = ni for / = 1, 
. . . , p + 1 and Ti(k + 2 + p) = m + k + 1. Since (JT Tl ( n ) ) n is c.i.d. , one has 



■ k p+i 

Y[f 3 (x m+J )Y[ gi (x ni ] 

.3=1 i=i 



E 



n fj (x m+j ) n 9i (x ni ) 9p+ i (x np+1 : 

j=l 1=1 



k p 

E II fj (x m+ j ) H gi (Xm )g P +i (x m +k+i ) 

-i=i «=i 

Let T2 be a finite permutation such that T2(j) = m + j for j = 1, . . . ,k, 
T2(k + Z) = Z for Z = 1, . . . ,p + 1 and T2(A; + 2 +p) = m + /c + l. Since (X T2 ( n )) n 
is c.i.d., one also has 

" k p+1 

e n^-pwii^) 

,j=l 1=1 

k p 

=e n/i^iiisi^w^i] 

-j=l 2=1 
" fe p 

= -E 1 IX fj (x m +j ) H 3i (X)g P +i {x m+k+ i ) 
,j=i i=i 

Hence, (15) follows from (14) with r = k + 1 and = <? p +i. □ 

3. Some CLTs for c.i.d. sequences. In this section stable convergence (in 
particular, convergence in distribution) of 

'l 



^[-J2f(X k )-L r 



k=l 
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is investigated for three different choices of the random centering L n . 

In all cases, our main tool is the following version of the martingale 
CLT; see Hall and Heyde (1980), Theorem 3.2, page 58. Let {Y nk :n>l,k = 
1, ...,k n } be an array of real square integrable random variables, where 
k n t oo, and for all n, let T n § C T n \ C • • • C T nkn C A be cr-fields with 
T nO = {0,n}. If: 

(i) &(Ynk) C T nk , E\Y nk \F n ,k-i\ = a.s., T nk C J" n+l fc , 

(ii) maxi< fe < fen \Y nk \ -> in probability, sup n £[maxi< fe < fcn Y n 2 fc ] < oo, 

(iii) Efc=i ^rifc — > L in probability, for some real random variable L, 

then X)jfc=i ^nfe converges stably. Precisely, let M(0, c) denote the Gaussian 
law with mean and variance c > 0, where Af(0,0) = S . Then, YX=i Ynk 
converges stably with representing measure M(0, L), that is, for each H G A 
with P{H) > 0, J2k=iYnk converges in distribution under P(-\H) to the 
probability law 

M0 = j M(^L(lo)){-)P(MH). 
Let us start with the case L n = V/-. 

Theorem 3.1 (CLT, case I). Suppose (X n ) n is c.i.d., f and f 2 are in 
H and there exists an integer m>0 such that (f(X n+m ) — Vf) n is c.i.d. 
Then 

W nJ = -^(/(Xi) + • • • + f(X n ) - nVf) 
y n 

converges stably with representing measure N(0,Vp — (Vf) 2 ). 



Proof. For each n> m, define 

Y nk = n- 1 / 2 {f{X k+m )-V f ) 

and T nk = a(Y nl , Y nk ) for k = 1, . . . , n - m. Since (f(X n+m ) - V f ) n is 
c.i.d., condition (i) holds. By Theorem 2.2, condition (iii) holds with L = 
Vp — {Vf) 2 . As to (ii), first note that it can be equivalently written as 



7) 



-1/2 



max \f(X k )\^0 in probability 



m<k<n 



and 



supn 1 E 



max / (X k ) 

m<k<n 



< OO. 
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Fix e > and put A nk = {\f(X k )\ > e^/n}. Then 

/ \ 1 n 

P max |/(X fc )|> ev ^ £ E[lA n J 2 {X k )} 



and 

p2 



sup — E 

n n 



max / {X k ) 

rn<k<n 



k=m+l 



1 n 

<sup- £ £[/ 2 (X fe )]<£[/ 2 (Xi)]. 
n n 

k=m+l 



If (X n ) n is exchangeable, then (/(X n+m ) — V/) n is exchangeable for all 
m, so that Theorem 3.1 applies. Generally, however, the assumption that 
(f(X n+m ) — Vf) n is c.i.d. for some m cannot be dropped. 

Example 3.2 (Example 1.2 continued). Let X n = YZ=i z k + U n , where 
(Z n ) n and (U n ) n are independent sequences of independent random vari- 
ables, Z n ~ A/"(0, 6 n — 6 n -l)> ~ A/"(0, c — 6 n ), with fro = and 6 n f c. Let / 
be the identity mapping, Qq = {0, ft} and Q n = cr(^i, f/i , . . . , Z n , U n ). Then 
(X n ) n is Q-c.i.d. Since X k — U k = J2i=i %i ~~ * %i a - s -i Theorem 2.2 im- 
plies 

i n 1 n 00 

y / = lim-EX fc = lim-E(X fc -[/ fc ) = EZ fc a.s. 

Then 



n IT, * — ' n 77, 

fc=l fc=l k=l 



\faW nJ = Y J {{n-k+l)Z k + U k )-nY^Z k = Y J {Uk-{k-l)Z k )-nY J Z k , 

k=l k=l k=l k>n 

so that W n j is Gaussian with 
1 n 

Vav[W nJ } = -J2(c-b k + (k- l) 2 (b k - 6 fc _i)) + n(c - 6 n ). 
n k=i 

Hence, if lim n n(c — b n ) = oo, then lim n Var[W n j] = oo. In that case, since 
W n j is Gaussian for all n, (W n j) n does not converge in distribution. 

Let us turn to the second type of random centering, that is, L n = - x 
J2 k =i E[f (X k )\Q k ^i]. This is perhaps the less interesting of our choices of 
L n , at least from the point of view of applications. Nevertheless, there are 
situations where such a choice of L n plays a role, for instance, in stochas- 
tic approximation, calibration and gambling; see Hanson and Russo (1981, 
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1986), Dawid (1982) and Berti and Rigo (2002). In any case, the following 
result is available (CLT, case II; we omit the straightforward proof). Let 

1 n 

B nJ = —J2(f(X k ) - E[f{X k )\G k -i]) for all / € H. 

If (X n ) n is £/-c.i.d. and / and f 2 are in H., then 

(B n f) n converges stably with representing measure Af(0, Vp — (V/) 2 ). 

(16) 

Finally, we consider the case L n = E[f(X n +i)\G n ]. From the point of view 
of statistical applications, mainly in Bayesian forecasting and discrete time 
filtering, this is perhaps the most significant case; see Section 1. Denote 

C nJ = 4=(/(*i) + • • • + f(X n ) - nE[f(X n+1 )\g n }) for all / e H. 



Theorem 3.3 (CLT, case III). Suppose (X n ) n is Q-c.i.d., f and f 2 are 
in H. and sup n E[C 2 j] < oo. If 

1 n 

M n = -J2(f(Xk)-kE[f(X k+1 )\g k ] + (k-l)E[f(X k )\g k . 1 }) 2 -^a 2 a.s. 

for some real random variable a 2 , then {C n j) n converges stably with repre- 
senting measure M(0, a 2 ). Moreover, if 

1 n 

- £ k 2 (E[f(X k+1 )\g k ] - EifiXk)^}) 2 - m probability, 

then B n j — C n j — ► in probability, and (C n j) n converges stably with rep- 
resenting measure M(0,Vp — (Vf) 2 ). 

Proof. Suppose first that M n — ► a 2 a.s. For n > 1 and k = 1, . . . ,n, 
define Y nk = E[W n j\G k ] - E[W n j\G k -i] and T nk = g k . Then condition (i) 
trivially holds, and since -E^Wnj |£/o] = a.s., one has C n j = E[W n j\G n ] = 
J2k=i Y n k- Hence, it is enough to check (ii) and (iii) with L = a 2 . On noting 
that 

V^Y n , k = f(X k ) - kE[f(X k+1 )\Gk] + (k - l)E[f{X k )\G k -i], 

one obtains Y2=i Y nk = M n ^ a 2 a.s. Since Y 2 n = M n - ^-M n _i -> a.s., 
it follows that maxj.< n \Y nk \ — ► a.s. Moreover, 

n 

< E[{E[W nJ \Gk] ~ E[W nJ \Gk-i]?] 
fc=i 



E 



max Y 2 h 

k<n nk 
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^EiEiWnjlGk] 2 - E[w nJ \g k . 



1?] 



k=l 



= E[E[W nJ \G n ] 2 ]=E[C 2 nJ ], 

so that (max-k<n\Ynk\)n is bounded in L 2 . Hence, (ii) and (iii) hold with 
L = a 2 , and this concludes the proof of the first part of the theorem. Next, to 
prove the second part, define D k = E[f(X k+ i)\Q k ] ~ E[f(X k )\Q k _i] an d sup- 
pose that ^ J2k=i k 2 Dk - > in probability. By (16), it is sufficient to see that 
Bn.f — C n j — > in probability, and a direct calculation shows that B n j — 
Cn J = Sfe=i kD k- For n > 1 and k = l,...,n, define Y nk = -^kD k and 
T nk = Qk - Then, (i) and (iii) hold with L = 0. In particular, {maxfc< n |l^,fc|} 2 < 
J2 k =i Y 2 k — ► in probability. Moreover, 

max|y nfe | < — max\f(X k ) - E[f(X k )\g k _ 1 ] - kD k \ 

k<n -v/n k<n 



+ ^max.\f(X k )-E[f(X h )\g k - 1 ]\, 

\Jn k<n 

and both terms in the right-hand side are bounded in L 2 . (Boundedness of 
the first term has been shown in the first part of the proof.) Hence, condition 
(ii) holds, and this implies Z)fc=i kD k = J2 k =i Ynk — > in probability. □ 

The assumption that (C n j) n is bounded in L 2 surely holds if {W n j) n 
is bounded in L 2 and, in turn, this is true if (f(X n+rn ) — Vf) n is c.i.d. for 
some m. In particular, Theorem 3.3 implies that {C n j) n converges stably 
whenever (X n ) n is exchangeable, E[f{X\) 2 } < oo and (M n ) n converges a.s. 
Here, it is tempting to conjecture that (M n ) n always converges a.s. in the 
exchangeable case, but we do not know whether this is true. 

We close this section by applying the previous results to some of the 
examples in Section 1. Example 3.4 shows that, for c.i.d. nonexchangeable 
sequences, (W n j) n , (B n j) n and (C n j) n can have quite different asymptotic 
behaviors. Example 3.5 deals with modified Polya urns, in the particular 
case where the extra balls d n are i.i.d. 



Example 3.4 (Example 1.2 continued). Let X n = X)J?=i Z k + U n , where 
(Z n ) n and (U n ) n are independent sequences of independent random vari- 
ables, Z n ~ A/"(0, b n — & n _i), U n ~ jV(0, c — b n ), with bo = and b n | c. Let / 
be the identity mapping, Q$ = {0, £1} and Q n = o(Z\ , U\ , . . . , Z n , U n ) . Then, 
as noted in Example 3.2, (X n ) n is Q-c.i.d., Vf = J2T=i^k a -S- and W n j is 
Gaussian with mean and Var[W n j] = ^ Y^k=ii c ~ ^k + (k — l) 2 (&fc — &fc-i)) + 
n(c — b n ). Suppose now that 

supn 2 (6 n — 6 n _i) < oo and n(c — b n ) — >u 
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for some u. Then, a direct calculation shows that ^J2k=i(^ ~ l) 2 (^fc — 
6 fc _i) u, and M n = I Efc=i(^fc - - ^k? -» u a.s. Thus, (W n ,/) n con- 
verges in distribution to AA(0,2u) and, by Theorem 3.3, (C n j) n converges 
in distribution to M(0,u). Finally, 



-. n * n / k \ 2 

1 x — > .. I 



2 



a.s. 



fe=l ' fc=l\i=l / \i=l / 

and thus (16) yields -E>n,/ — > in probability. 

Example 3.5 (Example 1.3 continued). Let (X n ) n and (d n ) n be as in 
Example 1.3, and let Qo = {0,Q} and Q n = cr(Xi, d\, . . . ,X n , d n , d n +i) for 
n > 1. Suppose that d n is independent of a(Xi,dj'.i < re, j < n) for all 
re > 1 [i.e., condition (i) holds] and the d n are identically distributed with 
E[d 2 ] < oo. As shown in Example 1.3, (X n ) n is G-c.i.d. Let / be the identity 
mapping. By standard but long calculations, it can be shown that (C n j) n is 
bounded in L 2 and M n —> S(V — V 2 ) a.s., where 5 = Var[<Zi]/ E[d\] 2 and V = 
lim n - J2k=i x k a.s. We refer to Berti, Pratelli and Rigo (2002) for details 
on such calculations. In any case, by Theorem 3.3, (C n j) n converges stably 
with representing measure JV"(0,<5(V — V 2 )). 

4. Uniform limit theorems. In Section 3, given a Q-c.i.d. sequence (X n ) n , 
convergence in distribution of 

W nJ = -U/(X!) + • • • + f(X n ) - nV f ), 



n 

Bnj = 4= E(/(^*) " E[f(X k )\G k ^]), 

v n fc= i 



C nJ = + • • • + /(X n ) - n£[/(X„ +1 )|<?„]) 



n 



has been investigated for a fixed function /. In this section W n := {W n j '■ f £ 
V}, B n := {B n j : f G 2?} and C n := {C n j : / G P} are seen as processes, in- 
dexed by some class PcH of functions, and their convergence in distribu- 
tion is analyzed in the path space under uniform distance. Note that W n , B n 
and C n all reduce to the usual empirical process whenever X n is independent 
of Q n -i for all n. 

For the sake of simplicity, we do not deal with a general Donsker-class T>, 
but we focus on the particular case where 

(E,S) = (R,B(R)) and © = {J ( _ 00it] :t €R}. 
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Hence, letting 

X = < x : x is a real cadlag function on R and lim x(t) = 

W n ,t = ^ / n,/ ( _ 00>t] , #n,f. = -Bn,/ ( _ 00jt] , C n) t = Cn./c-oo,*] , 

the paths of W n , B n and C n belong to X (up to modifications on P-null 
sets). 

Throughout, X is equipped with uniform distance. We refer to the theory 
of weak convergence developed by Hoffmann-j0rgensen, van der Vaart and 
Wellner; see van der Vaart and Wellner (1996). Let (Q' ,A! ,P') be a proba- 
bility space and Z : £1' — > X a random element of X. Say that Z is measurable 
if {Z G B} G .A' for all Borel sets B (Z X, and that Z is tic/M if Z is indis- 
tinguishable from a measurable random element with a tight probability 
distribution. If Z and Z' are both measurable and tight, Z ~ Z' if and only 
if they have the same finite-dimensional distributions. If Z is measurable 
and the Z n are arbitrary random elements of X, then Z n — ► Z in distribu- 
tion means E*[f(Z n )] — ► -E[/(Z)] for all / G C&(Af), where -E* denotes outer 
expectation. If Z is not measurable, but indistinguishable from a measur- 
able random element Z' , then Z n — > Z in distribution stands for Z n — ► Z' in 
distribution. Suppose the Z n are random processes on (£l,A,P) such that 
(Z nj j 1 , . . . , Z„, t t r ) converges in distribution for all ti, . . . ,t r G R. Then, for 
(Z n ) n to converge in distribution to a tight limit, it is sufficient that, for 
all e, 7/ > 0, there is a finite partition Ji, . . . ,I m of R by right-open intervals 
such that 

(17) lim sup PI max sup |Z njS — Z n) t| > e ) < 77; 

see van der Vaart and Wellner (1996), Theorems 1.5.4 and 1.5.6. 

When (X n ) n is C?-c.i.d., a possible limit in distribution for W n , B n and 
C n is a tight process whose distribution v is given by 

v{xeX:(x(ti),...,x(t r ))eA} = J N(0,Z(t 1 ,...,t r ))(A)dP 

for all t±, . . . ,t r G R and A G /3(R r ), where £(ti, . . . , t r ) is a random covari- 
ance matrix. One significant particular case is the following. Let G F denote 
a process, on some probability space, of the form 

Gf = G° F{t) for all t G R, 

where G° is a standard Brownian bridge on [0, 1] and F a random distribu- 
tion function, independent of G°, satisfying 

(F(ii),...,F(t r )) ~ (a(-oo,ti],...,a(-oo,t r ]) for all t 1 ,...,t r €R, 
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a being the directing measure of (X n ) n . Then, G F has finite-dimensional 
distributions of the type of v with 



Generally, G F can fail to be measurable. However, G F is measurable and 
tight whenever all the F-paths are continuous on A c for some fixed countable 
set AcR. 

Before stating results, we will give a technical lemma that is needed later 
on. It is presumably well known, and we provide a proof just to make the 
paper self-contained. Let us denote ||x|| = sup 4 \ x(t)\ for all x £ X . 

Lemma 4.1. Let Z be a tight random process with paths in X and 
E^\\Z\\\ < co. Then, for all e > 0, there is a finite partition Ii,...,I m of 
R by right- open intervals such that 



Proof. It can be assumed that Z is measurable. By tightness of Z and 
integrability of \\Z\\, there is a compact K such that < e/5. 

Let X\, . . . ,xn G K be such that K C Ufli-^i where Bi is the ball with 
center Xi and radius e/5. Take a partition Ii,...,I m of R by right-open 
intervals such that maxj, sup st£lk |xj(s) — Xi(t)\ < e/5 for all i = 1, . . . , N. 
Then 



and thus i^(maxfc sup s teIk \Z S — Z t \) < (3/5)e + 2£ , [I{- /Zl ^}||Z||] < e. □ 

Next, based on the results in Section 3, we give conditions for convergence 
in distribution of (B n ) n and (C n ) n . 

Theorem 4.2. // (X n ) n is Q-c.i.d. and (B n ) n meets condition (17), 
then B n — > G F in distribution and G F is tight. 

Proof. First note that G F is measurable when X is equipped with the 
ball o"-field U. Suppose the finite-dimensional distributions of (B n ) n converge 
weakly to those of G F . Then, since (B n ) n meets (17), B n — > Z in distribu- 
tion for some measurable tight process Z with the same finite-dimensional 
distributions of G F . Since Z is tight, Z £ A a.s. for some separable Borel 
set Ac X. Since AcU (by separability) and the distributions of G F and Z 
agree onZY, one obtains G F G A a.s. Let L = Ii&f^axF + I^ G F^^yH , where H 
is any fixed distribution function. Then L is a random distribution function 



E(ii, . . . ,t r ) = (F{U A tj)(l - F(U V tj))) 



l<i,j<r ' 



El max sup \Z S — Zt\ I < e. 

V k s,t£l k J 



/{^jmax sup \Z S - Z t \ < (3/5)e 
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indistinguishable from F, and G L is measurable and tight due to G having 
separable range and X is complete. Since G F is indistinguishable from G L , 
it follows that G F is tight, Z ~ G L and -B n — > G F in distribution. It remains 
to prove that the finite-dimensional distributions of {B n ) n converge weakly 
to those of G F . Fix t±, . . . ,t r ,ai, . . . ,a r G R, define / = £)i=i Oi^(-oo,ti] anc ^ 
note that Ya=i a i < ^ r f i nas distribution 

H(A) = J Af{0, Vp - (V f ) 2 )(A) dP, A G B(K). 

Hence, (16) implies 

r r 

^ diBnfo = B n j — > aiG^ in distribution. 

i=l i=l 

By letting a x , . . . , a r vary, one obtains (B n>tl , . . . , B n>tr ) ->■ (G£ , . . . , G£ ) in 
distribution. □ 

Convergence in distribution of (C n ) n needs more conditions. Furthermore, 
as suggested by Theorem 3.3, Examples 3.4 and 3.5, it may be that C n — ► C 
in distribution but the limit process C is not of the type of G^. Denote 

Qk(t) = I {Xk < t} - kE[I {Xh+1 < t} \g k ] + (k- l)E[I {Xk < t} \Q k _i] for t G R. 

Theorem 4.3. Suppose (X n ) n is Q-c.i.d., {C n ) n meets condition (17) 
and sup n E[C% t ] < oo for all t G R. If 

\ E ^ W{*h-i<*>I&] - £[/ { x fc < t} ie fe -i]) 2 - o 

(18) n k= i 

in probability Vt G R, 
i/ieri C n — > G^ in distribution and G F is tight. Moreover, if 
1 n 

(19) — 7 <7fc(sW(t) — ► °"( s )*) a - s - for all s,t EM., 

fe=i 

i/ien C n — > C in distribution, where C is a tight process whose distribution 
v is given by 

v{xeX:(x(t 1 ),...,x(t r ))eA} = J AA(0,S(ti,...,t r ))(A) dP 

for all t\, . . .,t r GR and A G B(W), with E(ti, . . . ,t r ) = (o"(ii,iy))i<ij<r- 

Proof. Arguing as in the proof of Theorem 4.2, it is enough to see that 
the finite-dimensional distributions of (C n ) n converge weakly to those of G^ 
under (18) and to those of C under (19). Fix t\, . . . ,t r , a±, . . . ,a r G R, define 
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/ = Ya=i a iI(-oo,ti] an d note that (C n j) n is bounded in L 2 . If (18) holds, 
then 

1 n 

- £ k 2 (E[f(X k+1 )\g k ] - ElfiXk)^}) 2 -> in probability, 
n k=i 

and Theorem 3.3 yields 

r r 

aiCnji = C n j — > ^2 a i^ti m distribution. 



i=l 



Similarly, if (19) holds, then M n — ► X)t j a i a j <7 (Uitj) a - s -) an d Theorem 3.3 
implies that {C n j) n converges in distribution to the probability law jjl on 
B(M) given by 

li(A) = J Ar^^aiajtrfatj^iA) dP 

G X:^2 ai x(U) e a\, A £ B(M). 



v< x 



i=l 



By letting ai,...,a r vary, it follows that {C n ^,. ■ ■ ,C n , tr ) ->• (<G£, . . . ,G£) 
in distribution under (18), and that (C n ,ti , • • • , Cn,t r ) — > (C^ , . . . , Ct r ) in dis- 
tribution under (19). □ 

Remark 4.4. Suppose (X n ) n is C?-c.i.d. and /C C Ti is a countable class 
of functions such that supy 6 £ |/| is in 7i. Since C n j = £?[W nj /|^ n ] a.s., one 
obtains 



E 



SUp|Cn,/| 


<E 


sup|W n! /[ 


L/ex: 




L/e/c J 



Likewise, a direct calculation shows that 



lim sup E 


sup 15^/1 


< 5 lim sup E 


sup |W n ,/| 


n 


L/e/c 


n 


L/e/c J 



By these inequalities, (B n ) n and (C n ) n can be connected to (W n ) n . In par- 
ticular, suppose that, for all e > 0, there is a finite partition I\, . .. ,I m of R 
by right-open intervals such that 



(20) 



lim sup -El max sup |W n s — W n t| ) < e. 



Then, (20) still holds with (B n ) n or (C n ) n in the place of (W n ) n and, thus, 
(B n ) n and (C n ) n meet condition (17). 
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Theorems 4.2 and 4.3 are general results on Q-c.i.d. sequences. In the 
exchangeable case, however, something more can be said. We close the paper 
by dealing with this case. 

Theorem 4.5. If (X n ) n is exchangeable and G F is tight, then W n — > G F 
in distribution. 

Theorem 4.5 can be proved by a standard application of de Finetti's 
representation theorem. We refer to Berti, Pratelli and Rigo (2002) for a 
proof. The assumption that G^ is tight, instead, needs two remarks. First, 
it can not be suppressed. Indeed, when G F is not tight, (W n ) n can fail to 
converge in distribution even if (X n ) n is exchangeable. An example is in 
Berti and Rigo (2004). Second, a tight version of G F is available if P{X\ = 
X 2 ) = or if P(Xi e A) = 1 for some countable icK. In fact, if (X n ) n 
is exchangeable and P{X\ = X 2 ) = 0, the random distribution function F 
can be taken to be continuous. Precisely, in some probability space, there 
are a standard Brownian bridge G° and a version of F, independent of 
G°, whose paths are continuous. Hence, G F = G° F is tight. Up to replacing 
"continuous" with "continuous on A c ," the same is true if P(X\ E A) = 1 
for some countable A. 

Finally, let us turn to B n and C n . Investigating their asymptotic behav- 
ior needs a little more than a straightforward application of de Finetti's 
theorem. 

Theorem 4.6. Suppose (X n ) n is exchangeable and G F is tight. Then, 
B n — > G F in distribution and (C n ) n is relatively sequentially compact. More- 
over, C n — > G F in distribution under condition (18) and C n — > C in distribu- 
tion under condition (19), where C is the tight process described in Theorem 
4.3. 

Proof. Suppose first that (W n ) n meets condition (20). Then, by Re- 
mark 4.4, (B n ) n and {C n ) n satisfy (17). By exchangeability, {C ni t) n is 
bounded in I? for all t. Thus, Theorems 4.2 and 4.3 yield B n — > G F in distri- 
bution, C n — > G F in distribution under (18) and C n — ► C in distribution un- 
der (19). Moreover, {C n ) n is relatively sequentially compact by Lemma 1.5.2 
and Theorems 1.3.9 and 1.5.6 of van der Vaart and Wellner (1996). Hence, it 
is enough to prove (20). If (X n ) n is i.i.d., then sup n £[||tUn|| 2 ] < c, where the 
constant c does not depend on the distribution of X\ ; see van der Vaart and Wellner 
(1996), pages 247 and 248. By de Finetti's theorem, sup n E[\\ W n \\ 2 } < c still 
holds if (X n ) n is exchangeable. It follows that (HVFnlDn is uniformly in- 
tegrable, and since W n — > G F in distribution (by Theorem 4.5), -E[||G F ||] = 
lim n £?[||W n ||] < oo. By Lemma 4.1, given e > 0, there is a partition Ii, . . . ,I m 
of R by right-open intervals such that £J(maxfc sup s tg/fc |Gf — G F \) < e. Let 
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h(x) = maxfc sup sig/fc \x(s) — x(t)\ for x £ X. Since /i is continuous, h(W n ) — > 
h(G F ) in distribution. Since /i(W n ) < 2||W n ||, (/i(W n )) n is uniformly inte- 
grable. Thus, limsup,„ E[h{W n )} = E[h(G F )] < e. □ 
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